Sentiment analysis of mental health disorder symptoms

ABSTRACT

Monitoring and analysis of a user&#39;s speech to detect symptoms of a mental health disorder by continuously monitoring a user&#39;s speech in real-time to generate audio data based, transcribing the audio data to text and analyzing the text of the audio data to determine a sentiment of the audio data is disclosed. A trained machine learning model may be applied to correlate the text and the determined sentiment to clinical information associated with symptoms of a mental health disorder to determine whether the symptoms are a symptom event. The initial determination may be transmitted to a second device to determine (and/or verify) whether or not the symptom event was falsely recognized. The trained machine learning model may be updated based on a response from the second device.

BACKGROUND

The present disclosure relates to the analysis of mental of mentalhealth disorder symptoms using sentiment analysis.

Mental health disorders including, for example, bi-polar disorder, maybegin during early childhood and may continue into adulthood. Bi-polardisorder, for example, may be characterized by intense mood swings thatinclude emotional highs (e.g. euphoric feelings) and lows (e.g.,depression). The mood shifts may occur only a few times a year or asoften as several times a week. The Child and Adolescent BipolarFoundation estimates that at least three quarters of a million Americanchildren and teens may suffer from bipolar disorder, although many arenot diagnosed. According to the National Institute of Mental Health, ina given year, bipolar disorder affects about 5.7 million Americanadults, or about 2.6% of the U.S. population 18 and older.

According to Center for Quality Assessment and Improvement in MentalHealth, bipolar disorder is frequently unrecognized, under diagnosed,and inappropriately treated. For example, patients generally do notrecognize or spontaneously report the symptoms of mania, e.g., a mentalillness marked by periods of great excitement, euphoria, delusions, andoveractivity, and hypomania, e.g., a mild form of mania, marked byelation and hyperactivity, as they view these periods as normalhappiness or well-being.

Forms, such as, for example, the Mood Disorder Questionnaire (MDQ) andthe Bipolar Disorder Symptoms & Functioning Monitoring Form, bothpublished by the center for quality assessment and improvement in mentalhealth (CQAIMH), have been designed to aid clinicians in the screeningof present and past episodes of mania and hypomania. However theaccuracy of the data found in these forms may be questionable becausethe forms may be incorrectly filled out.

BRIEF SUMMARY

One aspect of the present disclosure is a computer-implemented method,which includes monitoring speech by an audio input device, generatingaudio data by the audio input device based on monitored speech,transcribing the audio data to text, analyzing the text of the audiodata by a computing device to determine a sentiment, correlating thetext and determined sentiment to clinical information associated withone or more symptoms of a health disorder, and determining whether ornot the symptoms are a symptom event.

Other embodiments of the present invention include systems, and computerprogram products.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of the present disclosure, both as to its structure andoperation, can best be understood by referring to the accompanyingdrawings, in which like reference numbers and designations refer to likeelements and wherein:

FIG. 1 illustrates an exemplary system in accordance with the presentdisclosure.

FIG. 2 illustrates another exemplary system in accordance with thepresent invention.

FIG. 3 illustrates an exemplary method in accordance with the presentinvention.

FIG. 4 illustrates yet another exemplary system in accordance with thepresent invention.

DETAILED DESCRIPTION

In aspects of the present disclosure, the mental health of a patient maybe detected, monitored, and analyzed by continuously gathering dataabout the patient during their everyday life. The data may be analyzedto determine whether the patient may be exhibiting symptoms of a mentaldisorder, for example, mood changes in a patient may be symptoms ofbi-polar disorder. If the data indicates that the patient may beexhibiting symptoms of a disorder, the patient or a clinician may benotified so that services may be provided and the data may be stored ina database for later use.

FIG. 1 illustrates an exemplary system in accordance with the presentdisclosure. As depicted, a system 100 for monitoring a user, e.g., apatient, for symptoms of a mental disorder includes a computing device110, a server 150, and a database 170.

Computing device 110 includes at least one processor 112, memory 114, atleast one network interface 116, a display 118, an input device 120, anaudio input device 122 and may include any other features commonly foundin a computing device. In some aspects, computing device 110 may beembodied as a, a personal computer, laptop, tablet, smart device, smartphone, smart watch, smart wearable device, or any other similarcomputing device that may be used by a user or a clinician. In someaspects, for example, computing device 110 may include program modules(not depicted) configured to gather or receive data from the user andmay transmit the gathered or received data to server 150 for furtherprocessing and analysis (via other program modules (also not depicted)to determine whether the data indicates that the user may have a mentaldisorder. In some aspects, computing device 110 or another computingdevice (not depicted) may also be configured to receive data from server150 regarding an analysis result of a user. For example, such computingdevice(s) may be monitored by a user-authorized third party, such as auser-authorized clinician.

In some aspects, some or all of the processing and analysis may beperformed directly on computing device 110. In some aspects, forexample, computing device 110 may execute or implement an applicationthat performs monitoring of the user's mental health.

Processor 112 may include, for example, a microcontroller, FieldProgrammable Gate Array (FPGAs), or any other processor that isconfigured to perform various operations. Processor 112 may beconfigured to execute instructions as described below. Theseinstructions may be stored, for example, in memory 114. In some aspects,for example, memory 114 may store instructions to implement a mentalhealth monitoring application that implements any of the functionalitydescribed below.

Memory 114 may include, for example, non-transitory computer readablemedia in the form of volatile memory, such as random access memory (RAM)and/or cache memory or others. Memory 114 may include, for example,other removable/non-removable, volatile/non-volatile storage media. Byway of non-limiting examples only, memory 114 may include a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a portable compact disc read-only memory (CD-ROM), anoptical storage device, a magnetic storage device, or any suitablecombination of the foregoing.

Network interface 116 is configured to transmit and receive data orinformation to and from a server 150 or any other computing device viawired or wireless connections. For example, network interface 116 mayutilize wireless technologies and communication protocols such asBluetooth®, WIIFI (e.g., 802.11a/b/g/n), cellular networks (e.g., CDMA,GSM, M2M, and 3G/4G/4G LTE), near-field communications systems,satellite communications, via a local area network (LAN), via a widearea network (WAN), or any other form of communication that allowscomputing device 110 to transmit or receive information to or fromserver 150 or database 170.

Display 118 may include any display device that is configured to displayinformation to a user of computing device 110. For example, in someaspects, display 118 may include a computer monitor, television, smarttelevision, or other similar displays. In some aspects, display 118 maybe integrated into or associated with computing device 110, for example,as a display of a laptop, smart phone, smart watch, or other smartwearable devices, as a virtual reality headset associated with computingdevice 110, or any other mechanism for displaying information to a user.In some aspects, display 118 may include, for example, a liquid crystaldisplay (LCD), an e-paper/e-ink display, an organic LED (OLED) display,or other similar display technologies. In some aspects, display 118 maybe touch-sensitive and may also function as an input device 120.

Input device 120 may include, for example, a keyboard, a mouse, atouch-sensitive display 118, a keypad, a microphone, or other similarinput devices or any other input devices that may be used alone ortogether to provide a user with the capability to interact withcomputing device 110.

Audio input device 122 may include, for example, a microphone or othersimilar device that is configured to sense or gather audio information,e.g., speech, statements, or other verbal noises, made by the user andthe sensed or gathered audio information may be stored as audio data 124in memory 114. Non-limiting examples of audio input devices 122 mayinclude wearable directional microphones, wearable audio recorders,microphone necklaces, cell phone microphone, wireless earpieces, orother similar devices. In some aspects, audio input device 122 may be adirectional microphone oriented toward the user so as to capture theuser's speech and other sounds while filtering out backgroundenvironmental sounds.

In some aspects, speaker recognition technologies may be implemented bycomputing device 110 to differentiate between sounds received from theuser and sounds received from other sources. For example, the audioinformation captured by audio input device 122 may be filtered using thespeaker recognition technologies and the resultant output may be storedas audio data 124 in memory 114. For example, in some aspects, audiodata 124 may include only those sounds determined to be received fromthe user by the speaker recognition technologies. A few (non-limiting)examples of speaker recognition technologies include: Alexa VoiceServices™ (AVS), Watson SST™, Voice Note™, and other similar voicerecognition technologies.

In some aspects, the audio data 124 may include any captured sound,including those sources from the user or from other sources, and thespeaker recognition technologies may be applied by server 150 to theaudio data 124 to differentiate between the sounds made by the user andsounds made by other sources.

In some aspects, audio input device 122 may be a separate device fromcomputing device 110 and may communicate or transmit audio data 124 tocomputing device 110 for storage in memory 114. In some aspects, forexample, audio input device 122 may communicate with computing device110 via network interface 116 using any of the above describedcommunication protocols or any other similar communication protocol.

Server 150 includes a processor 152, memory 154, and a network interface156 that may include similar functionality as processor 112, memory 114,and network interface 116. In some aspects, server 150 may, for example,be any computing device, server, or similar system that is configured tointeract with or provide data to computing device 110. In some aspects,for example, server 150 may include one or more servers that areconfigured to perform cognitive analysis of the audio data 124 gatheredby audio input device 122 of computing device 110. For example, server150 may receive audio data 124 from computing device 110 and may analyzethe audio data 124 to determine whether any symptoms of mental healthdisorders are present in the audio data 124.

In some aspects, for example, server 150 may be configured to analyzeboth structured and unstructured data by applying advanced naturallanguage processing, information retrieval, knowledge representation,automatic cognitive reasoning, and machine learning technologies. Anexample system that may be used by server 150 to analyze audio data 124gathered from a user's speech includes IBM Watson®, a product ofInternational Business Machines Corporation of Armonk, N.Y. In someaspects, for example, server 150 may be configured to analyze asentiment of the audio data 124 and the duration that the sentimentlasts to determine when mood swings happen, how quickly, and for howlong.

Database 170 may store the results of the analysis performed by server150.

FIG. 2 illustrates another exemplary system in accordance with thepresent disclosure. With reference now to FIG. 2, the speech or otheraudible sounds made by a patient 202 may be continuously monitored bycomputing device 110. For example, audio input device 122 may monitorthe patient 202's speech and generate a continuous stream of audio data124. The audio data 124 may be temporarily or semi-permanently stored inmemory 114 and may be transmitted to server 150, e.g., via networkinterfaces 116 and 156.

In some aspects, server 150 may transcribe the audio data 124 to textusing known transcription technology 204. One (non-limiting) example ofsuch transcription technology, is the IBM Watson Speech To TextService™.

In some aspects, server 150 may analyze 206 the transcribed text of thepatient's speech to determine one or more attributes of the patient'sspeech. One (non-limiting) example of such analysis technology 206 isthe IBM Watson Tone Analyzer™. In some embodiments, server 150 maydetermine a tone or sentiment of the patient's speech. For example,server 150 may be configured to determine, e.g., whether the patient ishappy, angry, sad, or other moods/sentiment based on the determinedspeech attributes.

In some aspects, by way of further example, the analysis of thepatient's speech by server 150 may include monitoring the patient'sspeech attributes in real-time by evaluating a rate of the patient'sspeech in the audio data 124. For example, server 150 may determine apatient's mood or behavior, based on an analysis of the audio data 124.For example, it may be detected that the patient is in a depressed statewhen the patient's rate of speech is low, e.g., below a predetermined orpatient specific low threshold, and that the patient is in a manic statewhen the patient's rate of speech is high, e.g., above a predeterminedor patient specific high threshold. In some aspects, for example, abaseline rate of speech may be determined for the patient based onhistorical audio data of the patient and the high and low thresholds maybe determined based on the historical audio data.

In some aspects, for example, the determined speech attributes, e.g.,tone or sentiment, rate of speech, volume level (dB), or any changes inthe patient's speech attributes as compared to historically establishedbaseline behaviors of the patient, may be input into a machine learningmodule 208 which is trained to correlate descriptors from the convertedtext of the patient's speech, determined sentiment of the patient'sspeech, and volume level (dB) of the patient's speech with clinicalinformation, e.g., information on symptoms and corresponding mentalhealth diagnoses, using algorithms such as, for example, multiple linearregression, partial least squares fit, support vector machines, andrandom forest. In some aspects, for example, machine learning module 208may be configured to analyze the past pattern of a patient's mood swingsto predict future behaviors. For example, the machine learning module208 may be trained using supervised learning where the training inputsand testing feature vectors may have the same elements. The trainingvalues may be measured during a baseline session held with a patient fortraining purposes and the testing values may be generated duringreal-time monitoring of the patient.

In some aspects, the feature vector may be represented as a list of realnumbers or integers including, but not limited to rate of speech (Hz),volume of speech (dB), repeating speech patterns (number of repeatedwords per time interval), sentiment (e.g., anger=1, joy=2, sadness=3,etc.), mental health diagnosis (e.g., no diagnosis=1, mild diagnosis=2,severe diagnosis=3, etc.), physical manifestations of symptoms (e.g.,arm waving=1, pacing=2, etc.), medication (Lamictal=1, Seroquel=2,Abilify=3, Klonopin=4, etc.), treatments (behavior modification=1, musictherapy=2, group therapy=3, etc.), or other similar feature vectorelements.

In some aspects, the output of the selected classifier may be evaluatedin terms of the false acceptance rate vs. the false rejection rate, alsoknown as the equal error rate (EER). This accuracy percentage maydetermine the confidence level that the patient is exhibiting signs ofbipolar or other mental health disorder behavior at any point in time.In some aspects, the threshold may be determined on an individual basisfor each patient. In some aspects, until a baseline behavior for apatient is established, the threshold may be determined based onhistoric data from patients with a similar behavioral profile. Forexample, a mood swing can last for hours, days, or months, depending onthe individual patient.

Machine learning algorithms may also be used to determine which featureshave the most influence on the result. This information, combined withpatterns determined from tracking the machine learning accuracy resultsmay be used to predict future behavior. For example, if rapid speech hasbeen shown through machine learning to indicate a change toward bipolarbehavior for an individual, this behavior may be monitored as a highpriority for that patient.

In some aspects, the analyzed frequency and patterns of mood swingsfound in the audio data 124 may be used to evaluate the effectiveness ofmedications or treatments. For example, the outcome of the analysis ofthe audio data 124 may be compared to the analysis of historical audiodata, e.g., the baseline for the patient, to determine whether amedication or treatment is effective in reducing the patient's symptoms.

In some aspects, the output result of machine learning module 208, e.g.,a determination that a mood swing has occurred for the patient, may bestored or recorded in database 170, for later use. In some aspects, theoutput result of machine learning module 208 may be transmitted to acomputing device 110 of patient 202, a computing device 110 of aclinician 212, and/or both, for further analysis. In some aspects, forexample, the computing device 110 of a clinician 212 may receive anoutput result from the machine learning module 208, e.g., from server150, that indicates that the patient is exhibiting symptoms of a mentalhealth disorder. The computing device 110 may alert the clinician 212 ofthe results and in some aspects, may propose scheduling a checkup orcounseling session with the patient 202 to the clinician 212 to discussthe symptoms.

FIG. 3 illustrates an exemplary method in accordance with the presentdisclosure. With particular reference now to the example depicted inFIG. 3, exemplary method 300 continuously monitors a user, e.g., apatient, for symptoms of a mental health disorder.

At 302, audio input device 122 continuously monitors any sounds orspeech made by the user and generates audio data 124. In some aspects,audio input device 122 may transmit the audio data 124 to the computingdevice 110 of the user for storage in memory 114. In some aspects, audioinput device 122 may be included as a component of computing device 110and may store the audio data 124 in memory 114 directly. The audio data124 may be stored temporarily in memory 114, e.g., in a buffer, or maybe stored semi-permanently in memory.

At 304, the computing device 110 of the user transmits the audio data124 to server 150. In some aspects, the computing device 110 of the usermay continuously stream the audio data 124 to the server 150 inreal-time. In some aspects, some or all of the method may be performedby computing device 110

At 306, the server 150 transcribes the received audio data 124 to textusing a speech to text engine, e.g., using the IBM Watson Speech To TextService™.

At 308, server 150 analyzes the text of the audio data 124 to determinea mood or sentiment of the user's speech, e.g., using a sentimentanalysis tool such as the IBM Watson Tone Analyzer™.

In some aspects, for example, server 150 may analyze the text todetermine a mood of the patient's speech, analyze the text or audio data124 to determine a rate of the user's speech, or may determine any otherspeech characteristics of the user's speech that may be used todetermine whether the user is exhibiting symptoms of a mental healthdisorder.

In some aspects, the audio data 124 may also be analyzed, for example,to determine changes in the volume level (dB) of the user's speech, orother similar non-textual components of the user's speech.

At 310, the results of the analysis and in some aspects the audio data124 are transmitted to database 170 for storage. For example, asreal-time audio data 124 continues to accumulate, database 170 maycontinue to store the audio data 124 and corresponding analysis resultsfor later use as historical data for the user or for other users. Forexample, the audio data 124 and corresponding analysis results stored indatabase 170, e.g., historical data, may be later used to develop abehavioral baseline for the user that may be compared to the real-timeaudio data 124 to make determinations on whether the user is exhibitingany new or different symptoms, to determine the effectiveness of drugsor other treatments, or other similar uses. In some aspects, thehistorical data stored in database 170 may be used as a training corpusfor training a machine learning model 208 (FIG. 2) to assess users forsymptoms of mental health disorders. In some aspects, a plurality ofusers may have their audio data 124 and corresponding analysis resultsstored in database 170 and the audio data 124 and corresponding analysisresults for all of the users may be used as inputs for training machinelearning model 208 to detect symptoms of mental health disorders.

At 312 machine learning module 208 (FIG. 2) receives the results of theanalysis of the audio data 124 generated in real-time from the user,e.g., the tone, sentiment, mood, rate, etc., and generates an outputindicating whether or not a mood swing or other symptom has beendetected at 314. If no mood swing has been detected, the method returnsto 302 and continues to monitor the speech or sounds made by the userand generating audio data.

At 316, if a mood swing or other symptom has been detected, the time ofthe mood swing or other symptom may be flagged, e.g., a timestamp may begenerated based on a current time or a start time of the mood swing inthe audio data 124 may be flagged, and the mood swing or other symptommay be recorded with the flagged time in database 170. In someembodiments, the mood swing or other symptom may be referred to as asymptom event.

At 318, server 150 may transmit a message to a computing device 110 of aclinician associated with the user in response to the detection of amood swing or other symptom or in response to a symptom event beingrecorded in database 170. In some aspects, for example, the message mayinclude a reference to the audio data 124 in database 170, the resultsof the analysis of the audio data, the flagged time, and the output ofthe machine learning module 208.

At 320, server 150 receives from the clinician, e.g., via computingdevice 110 of the clinician, a message indicating whether the symptomevent is valid (or on the other hand, was falsely recognized). Forexample, the clinician may review the message and any of the audio data124, results of the analysis, flagged time, and output of the machinelearning module 208 to determine whether the symptom event is falselyrecognized. For example, in some aspects, the clinician may determinethat the audio data 124 does not indicate any symptoms of a mentalhealth disorder. In some aspects, the clinician may confirm that theaudio data 124 does indicate that the patient is experiencing symptomsof a mental health disorder.

At 322, server 150 determines whether the message received from thecomputing device 110 of the clinician indicates that the symptom eventwas falsely recognized or correctly recognized.

If the message indicates that the symptom event is falsely recognized,server 150 may update the machine learning module 208 in real-time basedon the indication at 326. For example, server 150 may use the indicationthat the symptom event was falsely recognized to further train machinelearning module 208 by correlating the audio data 124 and correspondinganalysis results with a confirmed non-symptom event outcome. The methodmay then proceed to 302 and audio input device 122 may continuemonitoring the user's speech and sounds and generating audio data 124.

If the message indicates that the symptom event is correctly recognized,server 150 may update the machine learning module 208 in real-time basedon the indication at 326. For example, server 150 may use the indicationthat the symptom event was correctly recognized to further train machinelearning module 208 by correlating the audio data 124 and correspondinganalysis results with a confirmed symptom event outcome. The method maythen proceed to 302 and audio input device 122 may continue monitoringthe user's speech and sounds and generating audio data 124.

The following example scenarios using system 100 (FIG. 1) and method 300(FIG. 3) will now be described.

In the first example scenario, a user named Judy is suffering frombi-polar disorder and receiving bi-polar medication. Judy is doing verywell with her bipolar medication and enjoys spending time with otherresidents in a rehabilitation facility. If Judy has a manic episode,however, she may become dangerous to both herself and the otherresidents at the facility. Staff at the rehabilitation facility is notavailable to watch her at all times.

Judy has a computing device 110, e.g., a smart watch, that continuouslymonitors and analyzes her words when she is in the common room. Forexample, her location may be identified by a GPS system associated withher smart watch. The audio data 124 from Judy's smart watch may betransmitted to a server 150 for analysis as described above. If server150 determines that a pattern in Judy's speech has changed, e.g., as aprecursor to a manic episode, a message may be sent to a nurse at therehabilitation facility providing details of the pattern change and anindication that Judy may need help. In some aspects, a message may alsobe sent to Judy indicating that she should remove herself from thecurrent situation.

In a second example scenario, a user named Joe is a senior high schoolstudent going through tremendous pressure to prepare and submit collegeapplication forms. Joe has a computing device 110, e.g., a smart watch,that continuously monitors and analyzes his words. The audio data 124from Joe's smart watch may be transmitted to a server 150 for analysisas described above. During analysis, server 150 may determine that Joehas experienced a period of extreme joy followed by an extreme sadnessand that this pattern has been occurring several times a week. A messagemay be transmitted from server 150 to Joe's family doctor providing thedoctor with the audio data 124 and the analysis results. Joe's familydoctor may then invite Joe to the clinic for further testing.

FIG. 4 illustrates yet another exemplary computer or processing systemin accordance with the present disclosure. Some aspects may implementany portion of system 100, computing device 110, server 150, database170, systems, methods, and computer program products described herein. Acomputer system is only one example of a suitable processing system andis not intended to suggest any limitation as to the scope of use orfunctionality of embodiments of the methodology described herein. Thesystem shown may be operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well-known computing systems, environments, and/orconfigurations that may be suitable for use with the processing systemmay include, but are not limited to, personal computer systems, servercomputer systems, thin clients, thick clients, handheld or laptopdevices, multiprocessor systems, microprocessor-based systems, set topboxes, programmable consumer electronics, network PCs, minicomputersystems, mainframe computer systems, and distributed cloud computingenvironments that include any of the above systems or devices, and thelike.

An exemplary computer system may be described in the general context ofcomputer system executable instructions. With reference now to FIG. 5,the computer system executable instructions can be embodied as one ormore program modules 10, being executed by processor(s) 12 of thecomputer system. Generally, program modules 10 may include routines,programs, objects, components, logic, data structures, and so on thatperform particular tasks or implement particular abstract data types.

In some aspects, the computer system may be practiced in distributedcloud computing environments where tasks are performed by remoteprocessing devices that are linked through a communications network(“cloud”). In a distributed cloud computing environment, program modulesmay be located in both local and remote computer system storage mediaincluding memory storage devices.

The components of computer system may include, but are not limited to,one or more processors or processing units 12, a system memory 16, and abus 14 that couples various system components including system memory 16to processor 12. The processor 12 may include one or more program(software) module(s) 10 that perform the methods described herein. Themodule 10 may be programmed into the integrated circuits of theprocessor 12, or loaded from memory 16, storage device 18, network 24and/or combinations thereof.

Bus 14 may represent one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronics Standards Association (VESA) local bus, andPeripheral Component Interconnects (PCI) bus.

Computer system may include a variety of computer system readable media.Such media may be any available media that is accessible by computersystem, and it may include both volatile and non-volatile media,removable and non-removable media.

Memory 16 can include computer system readable media in the form ofvolatile memory, such as random access memory (RAM) and/or cache memoryor others (sometimes referred to “system memory”). Computer system mayfurther include other removable/non-removable, volatile/non-volatilecomputer system storage media. By way of example only, storage system 18can be provided for reading from and writing to a non-removable,non-volatile magnetic media (e.g., a “hard drive”). Although not shown,a magnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 14 by one or more datamedia interfaces.

Computer system may also communicate with one or more external devices26 such as a keyboard, a pointing device, a display 28, etc.; one ormore devices that enable a user to interact with computer system; and/orany devices (e.g., network card, modem, etc.) that enable computersystem to communicate with one or more other computing devices. Suchcommunication can occur via Input/Output (I/O) interfaces 20.

Still yet, computer system can communicate with one or more networks 24such as a local area network (LAN), a general wide area network (WAN),and/or a public network (e.g., the Internet) via network adapter 22. Asdepicted, network adapter 22 communicates with the other components ofcomputer system via bus 14. It should be understood that although notshown, other hardware and/or software components could be used inconjunction with computer system. Examples include, but are not limitedto: microcode, device drivers, redundant processing units, external diskdrive arrays, RAID systems, tape drives, and data archival storagesystems, etc.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present disclosure are described herein with reference toflowchart illustrations and/or block diagrams. It will be understoodthat each block of the flowchart illustrations and/or block diagrams,and combinations of blocks in the flowchart illustrations and/or blockdiagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus (or system) to produce amachine, such that the instructions, when executed via the processor ofthe computer, or other programmable apparatus or system, create meansfor implementing the functions/acts specified in the flowchart and/orblock diagram block or blocks. These computer readable programinstructions may also be stored in a computer readable storage mediumthat can direct a computer, a programmable apparatus, system, and/orother devices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable apparatus, system or other device to causea series of operational steps to be performed by the computer, otherprogrammable apparatus, system or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, system, or other deviceimplement the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby systems that perform the specified functions or acts.

Although specific embodiments of the present invention have beendescribed, it will be understood by those of skill in the art that thereare other embodiments that are equivalent to the described embodiments.Accordingly, it is to be understood that the invention is not to belimited by the specific illustrated embodiments, but only by the scopeof the appended claims.

What is claimed is:
 1. A computer-implemented method, comprising:monitoring continuously and in real-time, by an audio input deviceassociated with a first party, speech from the first party; generating,by the audio input device, audio data based on monitored speech;transcribing, using a hardware processor of a computing device, theaudio data to text; analyzing, by the computing device, the text of theaudio data to determine a sentiment; training a model, using machinelearning, to correlate the text and the determined sentiment to clinicalinformation associated with one or more symptoms of a health disorder;storing the audio data, the text, the determined sentiment and an outputof the trained machine learning model in a database as historical data,wherein the trained machine learning model is trained based at least inpart on the historical data that is stored in the database; anddeveloping, over time, a behavioral baseline condition for the firstparty based on a history of audio data and corresponding text sentimentanalysis results; comparing a result of analyzing the text of thecurrent audio data against a baseline condition of said first party, anddetermining, based on said comparison, whether the first party isexhibiting a new or different symptom; and scheduling, via an interfacedevice, a checkup or appointment with a health care practitionerregarding said new or different symptom, wherein a symptom includes amood swing event, said method further comprising: analyzing, by thehardware processor, the sentiment of the speech and a duration of saidsentiment to identify a mood swing event including a time of occurrence,how quickly the mood swing event occurs, and for how long a mood swingevent occurs; determining, by the hardware processor, over time, apattern and frequency of each identified mood swing event; comparing adetermined pattern and frequency of the mood swing events against adatabase of known mood swing patterns; predicting, based on saidcomparing determined frequency and pattern of mood swing events, a moodswing occurrence exhibited by said first party in the future, andgenerating an output message via an interface, said message indicatingsaid predicted potential mood swing of said first party.
 2. The methodof claim 1, further comprising applying the trained machine learningmodel to correlate the text, the determined sentiment, and at least onespeech attribute of the audio data to the clinical informationassociated with one or more symptoms of a health disorder.
 3. The methodof claim 2, wherein the at least one speech attribute is selected from agroup consisting of: a volume level and a rate of speech.
 4. The methodof claim 1, further comprising: determining, by the hardware processor,whether or not symptoms are a symptom event, and upon determining thatthe symptoms are a symptom event, determining that the symptom event wasfalsely recognized; and updating the trained machine learning modelbased on the audio data, the text and the determined sentiment, with aconfirmed non-symptom event, in response to determining that the symptomevent was falsely recognized.
 5. The method of claim 1, furthercomprising: determining whether or not symptoms are a symptom event, andupon determining that the symptoms are a symptom event, determining thesymptom event was correctly recognized; and updating the trained machinelearning model based on the audio data, the text and the determinedsentiment with a confirmed symptom event, in response to determining thesymptom event was correctly recognized.
 6. The method of claim 1,further comprising: transmitting a message including an indication ofwhether or not the symptoms are a symptom event to a computing deviceassociated with a second user; receiving from the computing deviceassociated with the second user a message indicating whether or not thesymptom event was falsely recognized; and updating the trained machinelearning model based on the received message.
 7. The method of claim 1,further comprising: determine, using the hardware processor, based onsaid comparing a result of analyzing the text of the current audio dataagainst the baseline condition, whether a medication or treatmentadministered to said first party is effective in reducing the firstparty's symptoms.
 8. The method of claim 1, further comprising:training, using said hardware processor, said machine learning modelbased at least in part on training inputs comprising a testing featurevector, said testing feature vector having data representing one or moreof: a first party's rate of speech, a volume level of speech, arepeating speech pattern, a determined sentiment, a determined mentalhealth diagnosis, an indication of a physical manifestation of said oneor more symptoms, medication, and treatments applied to said firstparty.
 9. The method of claim 1, further comprising: transmitting, usingsaid hardware processor, a message including an indication of a detectedmood swing event associated with the first party to a second party;receiving, from a device associated with the second party, a messageindicating whether or not the mood swing event was correctly recognizedor falsely recognized; and updating the trained machine learning modelbased on the received message.